Adaptive light source

ABSTRACT

A method includes capturing a first image of a scene, detecting a face in a section of the scene from the first image, and activating an infrared (IR) light source to selectively illuminate the section of the scene with IR light. The IR light source includes an array of IR light emitting diodes (LEDs). The method includes capturing a second image of the scene under selective IR lighting from the IR light source, detecting the face in the second image, and identifying a person based on the face in the second image.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation-in-part of InternationalApplication No. PCT/EP2016/076360, filed Nov. 2, 2016, which claimspriority to U.S. Provisional Patent Application No. 62/253,580, filedNov. 10, 2015 and European Patent Application No. 16158004.8, filed Mar.1, 2016. International Application No. PCT/EP2016/076360, U.S.Provisional Patent Application No. 62/253,580, and European PatentApplication No. 16158004.8 are incorporated herein.

BACKGROUND

Semiconductor light-emitting devices including light emitting diodes(LEDs), resonant cavity light emitting diodes (RCLEDs), vertical cavitylaser diodes (VCSELs), and edge emitting lasers are among the mostefficient light sources currently available. Material systems currentlyof interest for manufacturing of high-brightness light emitting devicescapable of operation across the visible spectrum include Group III-Vsemiconductors, particularly binary, ternary, and quaternary alloys ofgallium, aluminum, indium, and nitrogen, also referred to as III-nitridematerials. Typically, III-nitride light emitting devices are fabricatedby epitaxially growing a stack of semiconductor layers of differentcompositions and dopant concentrations on a sapphire, silicon carbide,III-nitride, or other suitable substrate by metal-organic chemical vapordeposition (MOCVD), molecular beam epitaxy (MBE), or other epitaxialtechniques. The stack often includes one or more n-type layers dopedwith, for example, Si, formed over the substrate, one or more lightemitting layers in an active region formed over the n-type layer orlayers, and one or more p-type layers doped with, for example, Mg,formed over the active region. Electrical contacts are formed on the n-and p-type regions.

Due to their compact size and low power requirements, semiconductorlight-emitting devices are attractive candidates for light sources suchas camera flashes for hand-held, battery-powered devices, such ascameras and cell phones.

SUMMARY

In examples of the present disclosure, a method includes capturing afirst image of a scene, detecting a face in a section of the scene fromthe first image, and activating an infrared (IR) light source toselectively illuminate the section of the scene with IR light. The IRlight source includes an array of IR light emitting diodes (LEDs). Themethod includes capturing a second image of the scene under selective IRlighting from the IR light source, detecting the face in the secondimage, and identifying a person based on the face in the second image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram illustrating a system including at least onesemiconductor light-emitting device as a light source.

FIGS. 1B, 1C, 2A, 2B, and 2C illustrate methods of illuminating a sceneusing, for example, the system of FIG. 1A.

FIG. 3 illustrates a scene to be illuminated.

FIG. 4 illustrates a three dimensional (3D) map of the scene illustratedin FIG. 3.

FIG. 5 illustrates a flash intensity profile for the scene illustratedin FIG. 3.

FIG. 6 is a cross sectional view of one example of a light source.

FIG. 7 is a top view of an array of LEDs.

FIG. 8 is a cross sectional view of one LED in the array of FIG. 7.

FIG. 9 illustrates the scene that is illuminated in the examples in thefollowing figures.

FIGS. 10A, 11A, 12A, 13A, 14A, and 15A illustrate different illuminanceprofiles for the scene illustrated in FIG. 9.

FIGS. 10B, 11B, 12B, 13B, 14B, and 15B illustrate the amount of currentapplied to the LEDs in the array of FIG. 7 to generate the illuminanceprofiles illustrated in FIGS. 10A, 1A, 12A, 13A, 14A, and 15A.

FIGS. 16, 17A, and 17B illustrate the amount of current applied to theLEDs in the array of FIG. 7 to generate illuminance profiles for azooming application.

FIGS. 18A and 18B illustrate the amount of current applied to the LEDsin the array of FIG. 7 to generate an illuminance profile for awide-angle application.

FIG. 19 is a cross sectional view of an array of LEDs with individualoptics.

FIG. 20 illustrates a light source with multiple LED arrays.

FIG. 21 illustrates a scanning, narrow-beam light source.

FIG. 22 illustrates a light source with a matrix control element.

FIG. 23 illustrates a light source with light emitters that emit lightof different colors or color temperatures.

DETAILED DESCRIPTION

Though in the description below, embodiments of the invention aredescribed as camera flashes but other uses are contemplated and arewithin the scope of the invention.

One problem with all camera flashes is that objects close to the cameraare often overexposed while objects further from the camera do not getenough light. Embodiments of the invention include a light source suchas a camera flash, for example for a portable or battery-powered device,or for a larger non-battery-powered photo studio flash. Light sourcesaccording to embodiments of the invention may adapt their illuminanceprofiles to the scene and deliver the right amount of light to allobjects on the scene. The adaptive lighting system according toembodiments of the invention may include a semiconductor light sourcesuch as a semiconductor light-emitting device, thought any suitablelight may be used.

FIG. 1A illustrates an example of an adaptive lighting system 1according to embodiments of the invention. System 1 may be a smartphone, a tablet computer, a laptop computer, a desktop computer, acomputer monitor, a digital camera, or any suitable device. System 1 mayalso be a camera module for any suitable device. System 1 includes alight source 10 connected to a driver 12. Driver 12 supplies power tolight source 10, as described below. Light source 10 may be one or morearrays of infrared (IR) light emitting diodes (LEDs). Driver 12 isconnected to a microprocessor 14. Microprocessor 14 may include anapplication processor and a baseband processor, which may be connectedto a radio frequency (RF) transceiver 13 to make and receive calls. Anonvolatile memory 15 stores applications, such as a face detection andrecognition application 17, which microprocessor 14 loads to volatilememory 19 for execution. Microprocessor 14 receives input from an inputdevice 18 and a camera 11. System 1 may also include a 3D sensor 16.Input device 18 may be, for example, a user-activated input device suchas a button that a user presses to take a picture. Input device 18 maynot require a user input in some embodiments, such as in the case wherea picture is taken automatically. Input device 18 may be omitted in someembodiments.

3D sensor 16 may be any suitable sensor capable of making a 3D profileof the scene. prior to taking a picture. In some embodiments, 3D sensor16 may be a time of flight (ToF) camera. A ToF camera measures the timeit takes for light reflected from objects to travel back to the ToFcamera. The time may be used to calculate the distance to each object inthe scene. In some embodiments, 3D sensor 16 may be a structured lightsensor. A structured light sensor includes a projection device thatprojects a specially designed pattern of light on the scene. A cameraalso included in the structured light sensor measures the position ofeach part of the light pattern reflected from the objects of the sceneand determines the distance to these objects by triangulation. In someembodiments, 3D sensor 16 may be an auxiliary camera or cameraspositioned at a distance from each other in the body of the device. Bycomparing the position of the objects as seen by the auxiliary cameras,distances to each object can be determined by triangulation. In someembodiments, 3D sensor 16 is the autofocus signal of the main camera inthe device. While scanning the focus position of the camera lens, thesystem can detect at which positions which parts of the scene are infocus. A 3D profile of the scene is then built by translating thecorresponding lens positions into the distances to the objects that arein focus for these positions. A suitable autofocus signal is derived byconventional methods, for example by measuring the contrast or byutilizing phase detection sensors within the camera sensor. When phasedetection sensors are used, in some embodiments, for optimal functioningof the adaptive flash, the positions of individual phase detectionsensors may correspond to areas illuminated by separate segments of thelight source 10, as described below.

FIG. 1B is a flowchart of a method 300B for using system 1 (FIG. 1A) insome embodiments of the present disclosure. Method 300B may beimplemented by microprocessor 14 loading the instructions of application17 from nonvolatile memory 15 to volatile memory 19 and then executingthe instructions. Method 300B, and any method described herein, mayinclude one or more operations, functions, or actions illustrated by oneor more blocks. Although the blocks are illustrated in sequentialorders, these blocks may also be performed in parallel, and/or in adifferent order than those described herein. Also, the various blocksmay be combined into fewer blocks, divided into additional blocks,and/or eliminated based upon the desired implementation. Method 300B maybegin in block 304.

In block 304, microprocessor 14 causes camera 11 to capture a firstimage of a scene (corresponding to the camera's field of view). Block304 may be followed by block 306.

In block 306, microprocessor 14 detects a face in a section of the scenefrom the first image. Microprocessor 14 may use a face detectionalgorithm, such as the face detector in the Open Source Computer Vision(OpenCV) library, to locate the face in the scene. Block 306 may befollowed by block 318.

In block 318, microprocessor 14 activates light source 10 to selectivelyilluminate the section of the scene with IR light. For example,microprocessor 14 selects a group of the IR LEDs in light source 10based on the location of the face in the scene (the section of thescene), and causes driver 12 to only drive the selected IR LEDs. Block318 may be followed by block 320.

In block 320, microprocessor 14 causes camera 11 to capture a secondimage of the scene under the selective IR lighting generated in block318. Block 320 may be followed by block 322.

In block 322, microprocessor 14 detects the face in the second image.The face in the second image may have optimal exposure. Block 322 may befollowed by block 324.

In block 324, microprocessor 14 identifies or verifies a person based onthe face in the second image. Microprocessor 14 may use a facerecognition algorithm, such as the face recognizer in the OpenCVlibrary, to identify or verify the person based on the face in thesecond image. Method 300B may end in block 324.

FIG. 1C is a flowchart of a method 300C for using system 1 (FIG. 1A) insome embodiments of the present disclosure. Method 300B may be avariation of method 300B (FIG. 1B). Method 300B may begin in block 302.

In block 302, microprocessor 14 activates light source 10 to uniformlyilluminate a scene with IR light. For example, microprocessor 14 causesdriver 12 to drive each of the IR LEDs in light source 10 with a firstindividual current. Block 302 may be followed by block 304.

In block 304, microprocessor 14 causes camera 11 to capture a firstimage of the scene (corresponding to the camera's field of view). Block304 may be followed by block 306.

In block 306, microprocessor 14 detects a face in a section of the scenefrom the first image. Block 306 may be followed by block 318.

In block 318, microprocessor 14 activates light source 10 to selectivelyilluminate the section of the scene with IR light. For example,microprocessor 14 selects a group of the IR LEDs in light source 10based on the location of the face in the scene (the section of thescene), and causes driver 12 to drive each of the selected IR LEDs witha second individual current. The second individual current may begreater than the first individual current, and the total of the secondindividual currents (“second total current”) used to drive light source10 may be equal to the total of the first individual currents (“firsttotal current”) used to drive light source 10. Alternatively, the secondindividual current may be less than the first individual current, andthe second total current may be less than the first total current. Block318 may be followed by block 320.

In block 320, microprocessor 14 causes camera 11 to capture a secondimage of the scene under the selective IR lighting generated in block318. Block 320 may be followed by block 322.

In block 322, microprocessor 14 detects the face in the second image.The face in the second image may have optimum exposure. For example,when the face in the scene is underexposed in the first image, the facein the second image is illuminated with more IR light from the selectedIR LEDs aimed toward the face and driven with higher currents withoutincreasing the total current usage. In another example, when the face inthe scene is overexposed in the first image, the face in the secondimage is illuminated with less IR light from the selected IR LEDs aimedtoward the face and driven with less currents while decreasing the totalcurrent usage. Block 322 may be followed by block 324.

In block 324, microprocessor 14 identifies or verifies a person based onthe face in the second image. Method 300C may end in block 324.

FIG. 2A is a flowchart of a method 200A for using system 1 (FIG. 1A) insome embodiments of the present disclosure. Method 200A may be avariation of method 300B (FIG. 1B). Method 200A may begin in block 202.

In block 202 an input is generated, for example instructing that apicture be taken. In block 204 camera 11 takes a first image of thescene (corresponding to the camera's field of view) with flash (lightsource 10) turned off. In block 206 a face is detected in a section ofthe scene from the first image. In block 210 light source 10 is turnedon in low light output mode (typically called “torch mode”). At thistime the illuminance profile of the light source 10 is kept uniformwhere “uniform” means all portions of the scene are illuminated with aknown illumination profile. In block 212 an intermediate image iscaptured while light source 10 continues to be on with uniformilluminance profile and low brightness. In block 214 the face isdetected in the section of the image from the intermediate image. Inblock 216A the system (e.g., microprocessor 14) calculates the optimumbrightness for all parts of the face (or the overall scene) to achieveoptimal exposure. This can be done by subtracting the pixel brightnessvalues of the face (or the scene) in first image from the respectivepixel brightness values of the face (or the scene) in the second image,and scaling the differences to achieve the optimal exposure levels. Inblock 218 light source 10 are activated according to the illuminanceprofile calculated in block 216A. In block 220, a second image is takenby camera 11 while light source 10 is activated according to theilluminance profile calculated in block 216A. In block 222 the face isdetected in the second image. In block 224 a person is identified orverified based on the face in the second image.

FIG. 2B is a flowchart of a method 200B for using system 1 (FIG. 1A) insome embodiments of the present disclosure. Method 200B may be avariation of method 300B (FIG. 1B). Method 200B may begin in block 202.

In block 202, an input is generated, for example instructing that apicture be taken. In block 204 camera 11 takes a first image of thescene (corresponding to the camera's field of view) with flash (lightsource 10) turned off. In block 206 a face is detected in a section ofthe scene from the first image. In block 208, a 3D profile of the face(or the overall scene) is generated. For example, 3D sensor 16 maygenerate the 3D profile of the face (or scene), or 3D sensor 16 maysense data about the face (or the scene) and transmit the data tomicroprocessor 14, which may generate the 3D profile of the face (or thescene). In block 216B the system (e.g., microprocessor 14) calculatesthe optimum brightness for all parts of the face (or the scene) toachieve optimal exposure based on the 3D profile. For example,microprocessor may decide to provide additional IR lighting to parts ofthe face (or the scene) farther from camera 11. In block 218, based onthe calculation performed in block 216B, the scene is illuminated bylight source 10. In block 220, a second image is taken by camera 11while light source 10 is activated according to the illuminance profilecalculated in block 216A. In block 222 the face is detected in thesecond image. In block 224 a person is identified or verified based onthe face in the second image.

FIG. 2C is a flowchart of a method 200C for using system 1 (FIG. 1A) insome embodiments of the present disclosure. Method 200C may be avariation of method 300B (FIG. 1B). Method 200C may begin in block 202.

In block 202 an input is generated, for example instructing that apicture be taken. In block 204 camera 11 takes a first image of thescene (corresponding to the camera's field of view) with flash (lightsource 10) turned off. In block 206 a face is detected in a section ofthe scene from the first image. In block 208, a 3D profile of the face(or the overall scene) is generated. In block 210, light source 10 isturned on in low light output mode (typically called “torch mode”). Atthis time the illuminance profile of light source 10 is kept uniformwhere “uniform” means all portions of the scenes are illuminated. Inblock 212 an intermediate image is captured with light source 10 intorch mode. In block 214 the face is detected in the section of theimage from the intermediate image. In block 216C the system (e.g.,microprocessor 14) calculates the optimum brightness for all parts ofthe face (or the scene) to achieve optimal exposure based on the inputof the first image and the intermediate image taken and the 3D profileas described above in the text accompanying FIG. 2A and FIG. 2B. Inblock 218, based on the calculation performed in block 216B, the sceneis illuminated by light source 10. In block 220, a second image is takenby camera 11 while light source 10 is activated according to theilluminance profile calculated in block 216A. In block 222 the face isdetected in the second image. In block 224 a person is identified orverified based on the face in the second image.

In each of FIGS. 2A, 2B, and 2C, the input may be, for example, a userinput such as the user pushing a button, an input generated bymicroprocessor 14 (for example, if microprocessor 14 is programmed totake a picture at a predetermined time, or at a predetermined interval),or any other suitable input. FIG. 3 illustrates a scene to be capturedin a picture when the input is generated. The scene illustrated in FIG.3 includes a first person 30 in the foreground, and a second person 32in the background. This scene is selected for illustration purposesonly. Other scenes with multiple objects or persons at various distancesfrom the camera are also suitable for use of the present invention.

FIG. 4 illustrates the 3D profile for the scene illustrated in FIG. 3.In FIG. 4, the lighter shades correspond to shorter distance from thecamera, darker shades correspond to larger distance from the camera.Accordingly, the person 30 in the foreground has the lightest shading,indicating the person 30 is closest to the camera. The person 32 in thebackground has darker shading, indicating the person 32 is further fromthe camera. The background is black, indicating the furthest distancefrom the camera.

Objects located far from the flash may receive higher light intensity;objects located closer to the flash may receive less light. As iswell-known, illuminance of light decreases according to the inversesquare law of distance (illuminance˜1/distance²). With the 3D profile ofthe scene the required amount of light to distribute to which portionsof the scene can therefore be calculated. The algorithm calculating therequired intensity profile may also take into account the illuminancethat each of the objects in the scene receives from ambient light,information gathered with the capture of a first image, and may adjustthe amount of flash light accordingly. For example, objects 30 that arealready well-illuminated, for example because they are lightly coloredor reflective, may receive less light; objects 32 that are notwell-illuminated, for example because they are dark or not reflective,may receive more light than may be calculated solely based on theirdistance from the light source, as determined by the 3D profile.

Digital cameras and their image processors typically include facerecognition algorithms. In some embodiments, information from a facerecognition algorithm may be used to better illuminate faces compared toother objects. If there is not enough light to expose the completepicture well, faces benefit from more light. If the person is too closeand there is a danger of overexposure, this feature should be turnedoff, such that more light is not directed to the face. In someembodiments, the calculation of relative light from the 3D profile mayreduce the amount of light sent towards the eyes of the person tominimize “red eye” in the picture.

In some embodiments, the calculation of relative light from the 3Dprofile may identify parts of the scene (e.g., faces in the background)that are very far from the flash and cannot be properly illuminated. Aminimal amount of light is sent to these parts of the scene, in order tomaximize the amount of light sent towards the useful parts of the scene(e.g., faces in the foreground) and thus provide better use of availabledrive current capability.

In some embodiments, a user interface (for example, the touch screen ona smart phone) may allow a user control over the relative amount oflight sent to each portion of the scene. For example, the user may turnadaptive features of the flash on and off, may turn various parts of thealgorithm used to calculate the relative light from the 3D profile(described above) on and off, and may manually create flash accents onthe scene.

Several illumination modes are contemplated by embodiments of theinvention.

In some embodiments, in a first group of illumination modes,illumination from light source 10 is distributed across the scene toachieve the most homogenously useful illuminated picture. In particular,in some embodiments, overexposure is minimized: in the case whereforeground is well illuminated by ambient light, all light from lightsource 10 is directed to the background. In some embodiments, lightsource 10 acts as a fill in flash: in the case where the background iswell illuminated by ambient light, all light from light source 10 isdirected to foreground. In some embodiments, when the foreground and thebackground are evenly illuminated by ambient lighting, light from lightsource 10 may be send mostly to the background. In some embodiments, inthe case of a dark foreground, light from light source 10 illuminatesthe foreground just enough for a good picture, and the rest of the lightfrom light source 10 is sent to the background.

In some embodiments, in a second group of illumination modes, selectedobjects are illuminated. In particular, in some embodiments, incombination with face recognition, faces may be weighted highest forbest illumination. In some embodiments, in combination with facerecognition, background around faces (or other objects) may receive lesslight, for example to increase contrast between the illuminated face andthe background nearest the face. In some embodiments, selected zones ofthe scene are identified for example by a user input. Light from lightsource 10 may be directed only within the selected zone. Examples ofselected zones include zoomed-in images, or otherwise-identifiedportions of the scene. In some embodiments, for pictures of, forexample, business cards, light from light source 10 may be emitted witha very high uniformity level.

FIG. 5 illustrates light provided to the scene of FIG. 3 based on thecalculation illustrated in FIG. 4. In FIG. 5, lighter shadingcorresponds to more light from light source 10, and darker shadingcorresponds to less light from light source 10. As illustrated in FIG.5, more light is provided in region 42, corresponding to the backgroundperson 32, while less light is provided in region 40, corresponding toforeground person 30. Extra light is provided to the face 52 of theperson in the background. The least amount of light may be provided tothe background where neither person 30 nor person 32 appears (notshown).

FIGS. 6, 7, and 8 illustrate one example of a light source 10, which maybe used in system 1 illustrated in FIG. 1A. Any suitable light sourcemay be used and embodiments of the invention are not limited to thestructures illustrated in FIGS. 6, 7, and 8.

FIG. 7 is a top view of a square array 60 of LEDs 62. LEDs 62 may bewhite LEDs, IR LEDs, or a combination thereof. LEDs 62 may bemonolithically grown on a single substrate. Alternatively, LEDs 62 neednot be monolithically grown on a single substrate, but may be diced thenarranged on a mount such that neighboring LEDs are very close together.In some embodiments, the gap between LEDs 62 is less than ⅓ of adimension (for example, the width) of an individual LED 62. Though a 3×3square array is illustrated, any suitable number of LEDs may be used,and the array need not be square, it may be rectangular or any suitableshape. The size of individual LEDs may depend on several designparameters as, for example, building volume with optical lens included,field of view of the camera and number of LEDs in the array. Forexample, the array must include enough LEDs to illuminate the totalfield of view of the camera (i.e. the entire scene). For smart phoneapplications, the total width of the array may be no more than 2 mm insome embodiments.

For larger cameras, the width of the array may be no more than 10 mm insome embodiments. Though the individual LEDs are square, this is notrequired; rectangular LEDs or LEDs of any suitable shape may be used.

FIG. 6 is a cross sectional view of the light source 10. Array 60 ofLEDs 62 is positioned such that a majority of light extracted from array60 is emitted toward an optic 64. In the example illustrated, optic 64is spaced apart from array 60. Alternatively, optic 64 may be placed ontop of array 60. Optic 64 may be any suitable structure that collimatesthe light and directs light to the appropriate area of the scene. Optic64 may be, for example, a lens. multiple lenses, one or more Fresnellenses, one or more refractive lens, one or more total internalreflection lens elements, one or more reflectors, one or morecollimators, or any other suitable optic. In the examples below, optic64 is a Fresnel lens. Light source 10 may be in the shape of a box 66,with array 60 disposed on a bottom of the box, and optic 64 forming thetop of the box. Interior sidewalls 68 of box 66, any portion of thebottom that is not occupied by array 60, and any portion of the top thatis not occupied by optic 64, are part of the optical design, andtherefore may be reflective or light absorbing as appropriate.

FIG. 8 is a cross sectional view of one example of a single LED 62 inthe array illustrated in FIGS. 6 and 7. Any suitable LED may be used andembodiments of the invention are not limited to the structureillustrated in FIG. 8. In the device of FIG. 8, a majority of light isextracted from the LED through the growth substrate. Such a device maybe referred to as a flip chip device. The LED of FIG. 8 is formed bygrowing a III-nitride (e.g., gallium nitride for blue or UV LEDs) orIII-arsenide (e.g., gallium arsenide for IR LEDs) semiconductorstructure on a growth substrate 70 as is known in the art. Growthsubstrate 70 is often sapphire but may be any suitable substrate suchas, for example, a non-Ill-nitride material, SiC, Si, GaN, or acomposite substrate. A surface of the growth substrate on which theIII-nitride or III-arsenide semiconductor structure is grown may bepatterned, roughened, or textured before growth, which may improve lightextraction from the device. A surface of the growth substrate oppositethe growth surface (i.e. the surface through which a majority of lightis extracted in a flip chip configuration) may be patterned, roughenedor textured before or after growth, which may improve light extractionfrom the device.

The semiconductor structure includes a light emitting or active regionsandwiched between n- and p-type regions. An n-type region 72 may begrown first and may include multiple layers of different compositionsand dopant concentration including, for example, preparation layers suchas buffer layers or nucleation layers, which may be n-type or notintentionally doped, and n- or even p-type device layers designed forparticular optical, material, or electrical properties desirable for thelight emitting region to efficiently emit light. A light emitting oractive region 74 is grown over n-type region 72. Examples of suitablelight emitting regions include a single thick or thin light emittinglayer, or a multiple quantum well light emitting region includingmultiple thin or thick light emitting layers separated by barrierlayers. A p-type region 76 may then be grown over light emitting region74. Like n-type region 72, the p-type region 76 may include multiplelayers of different composition, thickness, and dopant concentration,including layers that are not intentionally doped, or n-type layers.

After growth of the semiconductor structure, a reflective p-contact 78is formed on the surface of p-type region 76. The p-contact 78 oftenincludes multiple conductive layers such as a reflective metal and aguard metal which may prevent or reduce electromigration of thereflective metal. The reflective metal is often silver but any suitablematerial or materials may be used. After forming p-contact 78, a portionof p-contact 78, p-type region 76, and active region 74 is removed toexpose a portion of n-type region 72 on which an n-contact 80 is formed.The n- and p-contacts 80 and 78 are electrically isolated from eachother by a gap 82 which may be filled with a dielectric such as an oxideof silicon or any other suitable material. Multiple n-contact vias maybe formed; n- and p-contacts 80 and 78 are not limited to thearrangement illustrated in FIG. 8. The n- and p-contacts may beredistributed to form bond pads with a dielectric/metal stack, as isknown in the art (not shown).

As described above, LEDs 62 in array 60 may be formed on a single wafer,then diced from the wafer as an array 60 with individual LEDs 62 in thearray still attached to a single growth substrate portion.Alternatively, many LEDs 62 may be formed on a single wafer, then dicedfrom the wafer, such that already-diced, individual LEDs are disposed ona mount to form array 60.

Substrate 70 may be thinned after growth of the semiconductor structureor after forming the individual devices. In some embodiments, thesubstrate is removed from the device of FIG. 8. A majority of lightextracted from the device of FIG. 8 is extracted through substrate 70(or the surface of the semiconductor structure exposed by removing thesubstrate 70). Embodiments of the invention are not limited to flip chipLEDs—any suitable device may be used.

A wavelength converting structure 84 may be disposed in the path oflight extracted from the light emitting device. The wavelengthconverting structure includes one or more wavelength convertingmaterials which may be, for example, conventional phosphors, organicphosphors, quantum dots, organic semiconductors, II-VI or III-Vsemiconductors, II-VI or III-V semiconductor quantum dots ornanocrystals, dyes, polymers, or other materials that luminesce. Thewavelength converting material absorbs light emitted by the LED andemits light of one or more different wavelengths. Unconverted lightemitted by the LED is often part of the final spectrum of lightextracted from the structure, though it need not be. The final spectrumof light extracted from the structure may be white, polychromatic, ormonochromatic. Examples of common combinations include a blue-emittingLED combined with a yellow-emitting wavelength converting material, ablue-emitting LED combined with green- and red-emitting wavelengthconverting materials, a UV-emitting LED combined with blue- andyellow-emitting wavelength converting materials, and a UV-emitting LEDcombined with blue-, green-, and red-emitting wavelength convertingmaterials. Wavelength converting materials emitting other colors oflight may be added to tailor the spectrum of light extracted from thestructure. The wavelength converting structure 84 may include lightscattering or light diffusing elements such as TiO₂.

In some embodiments, the wavelength converting structure 84 is astructure that is fabricated separately from the LED and attached to theLED, for example through wafer bonding or a suitable adhesive such assilicone or epoxy. One example of such a pre-fabricated wavelengthconverting element is a ceramic phosphor, which is formed by, forexample, sintering powder phosphor or the precursor materials ofphosphor into a ceramic slab, which may then be diced into individualwavelength converting elements. A ceramic phosphor may also be formedby, for example tape casting, where the ceramic is fabricated to thecorrect shape, with no dicing or cutting necessary. Examples of suitablenon-ceramic pre-formed wavelength converting elements include powderphosphors that are dispersed in transparent material such as silicone orglass that is rolled, cast, or otherwise formed into a sheet, thensingulated into individual wavelength converting elements, powderphosphors that are disposed in a transparent material such as siliconeand laminated over the wafer of LEDs or individual LEDs, and phosphormixed with silicone and disposed on a transparent substrate. Thewavelength converting element need not be pre-formed, it may be, forexample, wavelength converting material mixed with transparent binderthat is laminated, dispensed, deposited, screen-printed,electrophoretically deposited, or otherwise positioned in the path oflight emitted by the LEDs.

The wavelength converting structure 84 need not be disposed in directcontact with the LEDs as illustrated in FIG. 8; in some embodiments, thewavelength converting structure 84 is spaced apart from the LEDs.

The wavelength converting structure 84 may be a monolithic elementcovering multiple or all LEDs in an array, or may be structured intoseparate segments, each attached to a corresponding LED. Gaps betweenthese separate segments of the wavelength conversion structure 84 may befilled with optically reflective material to confine light emission fromeach segment to this segment only.

Interconnects (not shown) such as, for example, solder, stud bumps, goldlayers, or any other suitable structure, may be used to electrically andphysically connect LEDs 62 in array 60 to a structure such as a mount, aprinted circuit board, or any other suitable structure. The mount may beconfigured such that individual LEDs 62 may be individually controlledby driver 12 of FIG. 1A. The light emitted by the individual LEDs 62illuminates a different part of the scene. By changing the current toindividual LEDs, the light provided to a corresponding part of the scenecan be modified. The optimal illuminance profile for the scene,calculated as described above, may be obtained by providing anappropriate level of current to each LED 62.

In some devices such as mobile or battery-powered devices, the maximumamount of current available for adaptive lighting system 1 of FIG. 1A isoften limited by the capabilities of the device battery. When definingthe drive current levels to all the LEDs 62, system 1 typically takesinto account the maximum available current budget, and thereby definesthe drive current level for each LED 62 such that the total drivecurrent does not exceed the maximum, while the correct ratio ofintensity between the LEDs is maintained and total light output ismaximized.

FIG. 9 illustrates a scene to be illuminated in the examples illustratedbelow in FIGS. 10A, 11A, 12A, 13A, 14A, and 15A. The amount of currentprovided to each LED 62 for each example is illustrated in FIGS. 10B,11B, 12B, 13B, 14B, and 15B. The target 88, which may be a faceidentified by the dashed line in FIG. 9, requires more light than therest of the scene according to the calculation from the 3D profile asdescribed above. In each of FIGS. 10A, 11A, 12A, 13A, 14A, and 15A, theamount of light provided to a region decreases with increasing darknessof the shading. The light distributions illustrated in each figure maybe relative.

FIG. 10A illustrates how the scene is illuminated when all LEDs 62 aresupplied with the same amount of current, as illustrated in FIG. 10B.The center of the scene is brightly illuminated, while the outer edgesof the scene are less illuminated. Accordingly, the portion of thetarget near the center of the scene is more illuminated than the portionof the target near the edge of the scene.

FIG. 11A illustrates how the scene is illuminated when only three LEDsare supplied with current, each of the three receiving the same amountof current, while the other six LEDs receive no current. The three LEDs91, 92, and 93 supplied with current are the center LED, and the twobottom LEDs in the left-most column, as illustrated in FIG. 11B. Asillustrated in FIG. 11A, the right side of the scene, correspondingroughly to the target, is more brightly illuminated than the rest of thescene. The current density for LEDs 91, 92, and 93 in FIG. 11B may bethree times higher than the case illustrated in FIG. 10B, where all LEDsare supplied with equal current. The illuminance of the target in FIG.11A is about 1.6 times higher than the illuminance of the target in FIG.10A.

To obtain higher illuminance, fewer segments can be switched on, asillustrated in two examples shown in FIGS. 12A, 12B, 13A, and 13B.

FIG. 12A illustrates how the scene is illuminated when only two LEDs aresupplied with current, each receiving the same amount of current, whilethe other seven LEDs receive no current. The two LEDs 94 and 95 suppliedwith current are the two bottom LEDs in the left-most column, asillustrated in FIG. 12B. As illustrated in FIG. 12A, the right side ofthe scene, corresponding roughly to the target, is more brightlyilluminated than the rest of the scene. The illuminance of the target inFIG. 12A is greater than the illuminance of the target in FIG. 11A.

FIG. 13A illustrates how the scene is illuminated when only a single LEDis supplied with current while the other eight LEDs receive no current.The LED 96 supplied with current is the center LED in the left-mostcolumn, as illustrated in FIG. 13B. As illustrated in FIG. 13A, theright side of the scene, corresponding roughly to the target, is morebrightly illuminated than the rest of the scene, though the highlyilluminated spot is smaller than in FIGS. 12A and 11A. The illuminanceof the target in FIG. 13A is greater than the illuminance of the targetin FIG. 11A.

To improve the uniformity of illuminance across the entire target, thecurrent supplied to different LEDs may be varied, as illustrated in twoexamples shown in FIGS. 14A, 14B, 15A, and 15B.

FIG. 14A illustrates how the scene is illuminated when six LEDs aresupplied with varying levels of current and three LEDs receive nocurrent. The center LED 96 in the left column is supplied with fivetimes more current than the five LEDs 97, 98, 99, 100, and 101 whichsurround LED 96. The three LEDs in the right column receive no current,as illustrated in FIG. 14B. As illustrated in FIG. 14A, the right sideof the scene, corresponding roughly to the target, is more brightlyilluminated than the rest of the scene. The illuminance of the target ismore uniform than in, for example, FIG. 13A.

FIG. 15A illustrates how the scene is illuminated when four LEDs aresupplied with varying levels of current and five LEDs receive nocurrent. The center LED 102 in the left column is supplied with fourtimes more current than the bottom LED 105 in the center column, andwith twice as much current as the center LED 104 and the bottom LED 103in the left column. The top row of LEDs and the LEDs in the right columnreceive no current, as illustrated in FIG. 15B. As illustrated in FIG.15A, the right side of the scene, corresponding roughly to the target,is more brightly illuminated than the rest of the scene. The illuminanceof the target is more uniform than in, for example, FIG. 13A.

FIGS. 16, 17B, and 18B illustrate how current may be applied to array 60of LEDs 62 in FIG. 6, for zoom and wide angle applications. When acommand to zoom in the camera lens is received, LEDs near the center ofthe array receive more current, as illustrated in FIGS. 16 and 17B. FIG.17A illustrates how the scene is illuminated when the LEDs are suppliedwith varying levels of current as illustrated in FIG. 17B.

When a command to zoom out the camera lens is received, LEDs near theedge of the array receive more current, as illustrated in FIG. 18B. FIG.18A illustrates how the scene is illuminated when the LEDs are suppliedwith varying levels of current as illustrated in FIG. 18B.

In FIG. 16, for a zoom application, just the center LED 110 is suppliedwith current, while the eight LEDs surrounding the center LED receive nocurrent. The center of the scene will be brightly illuminated, while theedges of the scene will receive less light. Illuminance at the center ofthe scene may be increased by 2.2 times over the center of the scene inFIG. 10A, where all nine LEDs receive equal current.

In FIG. 17B, for a zoom application, the center LED 111 is supplied withtwice as much current as LEDs 112, and four times as much current asLEDs 114. The center of the scene is more illuminated than the edges ofthe scene. Illuminance at the center of the scene may be increased by1.15 times over the center of the scene in FIG. 10A, where all nine LEDsreceive equal current.

In FIG. 18B, for a wide-angle application, the eight LEDs 118 at theedges of the array receive equal current, while the center LED 116receives no current. Illuminance at the center of the scene may bereduced to 0.85 times the illuminance at the center of the scene in FIG.10A, where all nine LEDs receive equal current.

Adaptive lighting system 1 of FIG. 1A may be used to illuminate multipletargets (e.g., faces), by providing current to only the LEDscorresponding to each target, or by providing more current to the LEDscorresponding to each target. Adaptive lighting system 1 may be used toreduce overexposure in a scene containing elements that are close to thecamera and far from the camera, by providing current to only the LEDscorresponding to the elements far from the camera, or by providing morecurrent to the LEDs corresponding to the elements far from the camera.

The illuminance values given for the examples above are calculated forthe illustrated 3×3 array with a single Fresnel lens. The light outputof each LED in the examples above can be controlled by the drivercurrent of the LED, or by pulse duration with a fixed current.

FIGS. 19, 20, 21, 22, and 23 illustrate alternative light sources.

In the light source of FIG. 19, each LED 62 in the array has anindividual optic 122, rather than a single optic for the entire array,as illustrated in FIG. 6. Each optic 122 directs light from its LED to aspecific portion of the scene. Optics 122 may be any suitable opticincluding, for example, lenses, dome lenses, Fresnel lenses, reflectors,total internal reflection lenses, or any other suitable structure.Optics 122 need not be identical; different optics may be used fordifferent LEDs 62 in the array.

The light source of FIG. 20 includes multiple LED arrays with multipleoptical elements. For example, FIG. 20 illustrates two 3×3 arrays, eachwith a single corresponding Fresnel lens. More or fewer arrays may beused, and the arrays are not limited to the device illustrated. In someembodiments, each array illuminates a part of the scene. Array 124 inFIG. 20 illuminates the top 128 of the scene, while array 126illuminates the bottom 130 of the scene. In some embodiments, the arraysilluminate overlapping parts of the scene, in order to provide morelight to the overlapping parts. For example, the arrays may overlap inthe center of the scene, which may be a part of the scene that oftenrequires more light than the edges.

The light source of FIG. 21 uses a narrow-beam light emitting devicesuch as, for example, a laser. The light source of FIG. 21 includes alaser 140 with a wavelength converting element 142 disposed in the pathof the light from the laser. Focusing optics 144 may create a light beamof the desired size. The beam is incident on a first scanning mirror146, and a second scanning mirror 148, before being incident on thescene 150. The scanning mirrors may be moved such that the light beamscans the entire scene, while the driver controls the intensity of thelight source, such that different parts of the scene may receivedifferent amounts of light. When the beam scans parts of the scenerequiring higher intensity, the current supplied to the laser increases;when the beam scans parts of the scene requiring lower intensity, thecurrent supplied to the laser decreases.

The light source of FIG. 22 includes a matrix control element, such as adigital micromirror switching device or a multi-segment liquid crystaldisplay. Light from an LED or laser 152 illuminates the matrix controlelement 154. The intensity of the reflected or transmitted light ismodified depending on the calculated illuminance profile. The reflectedor transmitted light from the matrix switching element 154 is projectedonto the scene 156. Matrix switching element 154 may have many smallmirrors as pixels. The orientation of each mirror can be changed to tunethe intensity at each pixel. The orientation of the mirrors may also beused to create brighter regions, by overlapping the beams from differentmirrors.

The light source of FIG. 23 is color tunable. The light source of FIG.23 includes two arrays, 160 and 162, which are arranged to emit beams166 and 168, respectively, which overlap when they illuminate the scene164. Though two arrays like the array illustrated in FIG. 6 areillustrated, other suitable light emitters may be used. The system mayinclude 3 or more arrays with different emission spectra. Arrays 160 and162 emit different colors of light. For example, arrays 160 and 162 mayboth emit white light, though array 160 may emit white light with adifferent color temperature than array 162—i.e., one of array 160 andarray 162 emits warm white light. For example, the array that emits warmwhite light may emit light with a color temperature as low as 1700 K,and the array that emits cool white light may emit light with a colortemperature as high as 10000 K. The difference in color temperaturebetween the two arrays may be at least 1000 K in some embodiments, atleast 2000 K in some embodiments, at least 3000 K in some embodiments,and at least 4000 K in some embodiments. Alternatively, arrays 160 and162 may emit different monochromatic colors of light. The appropriatecurrent supplied to each LED in each array is calculated such that thesum of light from arrays 160 and 162 has the appropriate illuminance andcolor temperature for each portion of the scene. Arrays (or other lightemitters) emitting additional colors or color temperatures of light maybe added.

In some embodiments, LEDs emitting multiple spectra may be combined in asingle, interleaved array, with a single optic as illustrated in FIG. 6or with individual optics as illustrated in FIG. 19. LEDs of differentcolors may be arranged in groups, each group illuminating a portion ofthe scene, each group including at least one LED of each differentcolor.

The color tunable light source described above may be used to illuminatedifferent parts of the scene with light of different correlated colortemperature (CCT). For example, a color tunable light source may be usedto equalize the CCT of different ambient illuminants. The sections ofthe scene with low CCT ambient light may be illuminated with higher CCTlight, while the sections of the scene with high CCT ambient light maybe illuminated with lower CCT light.

In some embodiments, light source 10 may be used with different cameras.For example, a smart phone may have multiple cameras, or different smartphone models may use different cameras. The cameras may each have aspecific field of view, for which the flash for that camera is tuned(for example, tuned to provide a minimum level of illumination in thecorner of the field of view). Accordingly, for a conventional flash,each camera requires a separate flash that is tuned to that camera'sfield of view. With adaptive lighting system according to embodiments ofthe invention, a default current distribution for each camera could bedefined and selected when that camera is selected, such that a singlelight source may be used for multiple cameras. The default for eachcamera may be modified according to the scene being photographed, asdescribed in the embodiments above.

Though in the examples above the semiconductor light emitting device areIII-nitride LEDs that emit blue or UV light and III-arsenide LEDs thatemit IR light, semiconductor light emitting devices besides LEDs such aslaser diodes and semiconductor light emitting devices made from othermaterials systems such as other III-V materials, III-phosphide, II-VImaterials, ZnO, or Si-based materials may be used.

Having described the invention in detail, those skilled in the art willappreciate that, given the present disclosure, modifications may be madeto the invention without departing from the spirit of the inventiveconcept described herein. In particular, different elements fromdifferent examples or embodiments may be combined. It is not intendedthat the scope of the invention be limited to the specific embodimentsillustrated and described.

1: A method, comprising: capturing a first image of a scene; detecting aface in a section of the scene from the first image; activating aninfrared (IR) light source to selectively illuminate the section of thescene with IR light, the IR light source comprising an array of IR lightemitting diodes (LEDs); capturing a second image of the scene underselective IR lighting from the IR light source; detecting the face inthe second image; and identifying a person based on the face in thesecond image. 2: The method of claim 1, prior to said capturing thefirst image of a scene, further comprising activating the IR lightsource to uniformly illuminate the scene with IR light by driving eachof the IR LEDs in the array with a first individual current. 3: Themethod of claim 2, wherein said activating the IR light source toselectively illuminate the section of the scene with IR light comprisesselecting a group of IR LEDs in the array and driving each of the IRLEDs in the group with a second individual current that is greater thanthe first individual current. 4: The method of claim 3, wherein a firsttotal current comprising the sum of the first individual currents and asecond total current comprising the sum of the second individualcurrents are equal. 5: The method of claim 2, wherein said activatingthe IR light source to selectively illuminate the section of the scenewith IR light comprises selecting a group of IR LEDs in the array anddriving each of the IR LEDs in the group with a second individualcurrent less than the first current, a first total current comprisingthe sum of the first individual currents being greater than a secondtotal current comprising the sum of the second individual currents. 6:The method of claim 1, wherein said capturing the first image of a sceneoccurs under ambient lighting. 7: The method of claim 6, prior to saidactivating the IR light source to selectively illuminate the section ofthe scene with IR light, further comprising: activating the IR lightsource to uniformly illuminate the scene with IR light capturing anintermediate image of the scene under uniform IR lighting from the IRlight source; detecting the face in the intermediate image; andcalculating an amount of IR light to illuminate the section of the scenebased on brightness values of the face in the first and the intermediateimages. 9: The method of claim 7, wherein said calculating the amount ofIR light to illuminate the section of the scene is based on a differencebetween corresponding brightness value of the face from the first andthe intermediate images. 10: The method of claim 1, prior to saidactivating the IR light source to selectively illuminate the section ofthe scene with IR light, further comprising: creating a threedimensional profile of the face; and calculating an amount of IR lightto illuminate the section of the scene based on the first image and thethree dimensional profile of the face. 11: The method of claim 1, priorto said activating the IR light source to selectively illuminate thesection of the scene with IR light, further comprising: creating a threedimensional profile of the face; activating the IR light source touniformly illuminate the scene with IR light capturing an intermediateimage of the scene under uniform IR lighting from the IR light source;detecting the face in the intermediate image; and calculating an amountof IR light to illuminate the section of the scene is based brightnessvalues of the face in the first and the intermediate images and thethree dimensional profile of the face. 12: The method of claim 1,wherein a first LED in the array emits IR light of a different emissionspectrum than IR light emitted by a second LED in the array. 13: Themethod of claim 1, wherein the array is a first array of IR LEDs, the IRlight source further comprises a second array of IR LEDs, the first andthe second arrays of IR LEDs illuminate a same portion or differentportions of the face. 14: The method of claim 13, wherein the first LEDin the array emits IR light of a different emission spectrum than IRlight emitted by the second LED in the array. 15: The method of claim 1,wherein said identifying a person based on the face in the second imagecomprises comparing the face in the second image against facial imagesin a database and finding a match. 16: A system, comprising: a camerahaving a field of view; an infrared (IR) light source comprising anarray of IR light emitting diodes (LEDs); nonvolatile memory storinginstructions for a face detection and recognition application; amicroprocessor encoded with instructions for: capturing, with thecamera, a first image of a scene; detecting a face in a section of thescene from the first image; activating the IR light source toselectively illuminate the section of the scene with IR light, the IRlight source comprising an array of IR light emitting diodes (LEDs);capturing, with the camera, a second image of the scene under selectiveIR lighting from the IR light source; detecting the face in the secondimage; and identifying a person based on the face in the second image.17: The system of claim 16, wherein the microprocessor is furtherencoded with instructions for, prior to said capturing the first imageof a scene, activating the IR light source to uniformly illuminate thescene with IR light by driving each of the IR LEDs in the array with afirst individual current. 18: The system of claim 17, wherein saidactivating the IR light source to selectively illuminate the section ofthe scene with IR light comprises selecting a group of IR LEDs in thearray and driving each of the IR LEDs in the group with a secondindividual current that is greater than the first individual current.19: The system of claim 18, wherein a first total current comprising thesum of the first individual currents and a second total currentcomprising the sum of the second individual currents are equal. 20: Thesystem of claim 17, wherein said activating the IR light source toselectively illuminate the section of the scene with IR light comprisesselecting a group of IR LEDs in the array and driving each of the IRLEDs in the group with a second individual current less than the firstindividual current, a first total current comprising the sum of thefirst individual currents being greater than a second total currentcomprising the sum of the second individual currents. 21: The system ofclaim 16, wherein said capturing the first image of a scene occurs underambient lighting. 22: The system of claim 21, wherein the microprocessoris further encoded with instructions for, prior to said activating theIR light source to selectively illuminate the section of the scene withIR light: activating the IR light source to uniformly illuminate thescene with IR light capturing an intermediate image of the scene underuniform IR lighting from the IR light source; detecting the face in theintermediate image; and calculating an amount of IR light to illuminatethe section of the scene based on brightness values of the face in thefirst and the intermediate images. 23: The system of claim 22, whereinsaid calculating the amount of IR light to illuminate the section of thescene is based on a difference between corresponding brightness value ofthe face from the first and the intermediate images. 24: The system ofclaim 16, wherein the microprocessor is further encoded withinstructions for, prior to said activating the IR light source toselectively illuminate the section of the scene with IR light: creatinga three dimensional profile of the face; and calculating an amount of IRlight to illuminate the section of the scene based on the first imageand the three dimensional profile of the face. 25: The system of claim16, wherein the microprocessor is further encoded with instructions for,prior to said activating the IR light source to selectively illuminatethe section of the scene with IR light: creating a three dimensionalprofile of the face; activating the IR light source to uniformlyilluminate the scene with IR light capturing an intermediate image ofthe scene under uniform IR lighting from the IR light source; detectingthe face in the intermediate image; and calculating an amount of IRlight to illuminate the section of the scene is based brightness valuesof the face in the first and the intermediate images and the threedimensional profile of the face. 26: The system of claim 16, wherein afirst LED in the array emits IR light of a different emission spectrumthan IR light emitted by a second LED in the array. 27: The system ofclaim 16, wherein the array of IR LEDs is a first array of IR LEDs, theIR light source further comprises a second array of IR LEDs, the firstand the second arrays of IR LEDs illuminate a same portion or differentportions of the face. 28: The system of claim 27, wherein the first LEDin the array emits IR light of a different emission spectrum than IRlight emitted by the second LED in the array. 29: The system of claim16, wherein said identifying a person based on the face in the secondimage comprises comparing the face in the second image against facialimages in a database and finding a match. 30: A non-transitory,computer-readable storage medium encoded with instructions executable bya microprocessor to selectively illuminate a scene, the instructionscomprising: capturing a first image of a scene; detecting a face in asection of the scene from the first image; activating an infrared (IR)light source to selectively illuminate the section of the scene with IRlight, the IR light source comprising an array of IR light emittingdiodes (LEDs); capturing a second image of the scene under selective IRlighting from the IR light source; detecting the face in the secondimage; and identifying a person based on the face in the second image.